SKEW SCHUBERT POLYNOMIALS 



CRISTIAN LENART AND FRANK SOTTILE 

Abstract. We define skew Schubert polynomials to be normal form (polynomial) represen- 
tatives of certain classes in the cohomology of a flag manifold. We show that this definition 
extends a recent construction of Schubert polynomials due to Bergeron and Sottile in terms 
of certain increasing labeled chains in Bruhat order of the symmetric group. These skew 
Schubert polynomials expand in the basis of Schubert polynomials with nonnegative integer 
coefficients that are precisely the structure constants of the cohomology of the complex flag 
variety with respect to its basis of Schubert classes. We rederive the construction of Berg- 
eron and Sottile in a purely combinatorial way, relating it to the construction of Schubert 
polynomials in terms of rc-graphs. 



Introduction 

Skew Schur polynomials S\/ M (xi, . . . ,Xk) play an important and multi-faceted role in alge- 
braic combinatorics. For example, they are generating functions for the Littlewood- Richardson 
coefficients ,. 

(1) S x/ ^(x 1 ,...,x k ) = ^2(^ v S v (xi,...,x k ). 

They are also generating functions for Young tableaux of shape A//x 

(2) Sx/^xx,. . . ,x k ) = ^x T , 

where the sum is over all Young tableaux T of shape X/fi and x T is a monomial associated 
to T. The relationship between these two very different facets of skew Schur polynomials 
involves the combinatorics of Young tableaux [20, Chapter 7, Appendix 1]. Young tableaux 
can be thought of as increasing labeled chains in Young's lattice, with covers labeled by pairs 
(k,l), where k is the entry in the box corresponding to the cover, and I the content of that 
box (that is, the difference between its column and row); labels are ordered lexicographically. 
We introduce skew Schubert polynomials, which are the Schubert polynomial analogs of skew 
Schur polynomials, in the sense that they generalize the two properties mentioned above. 

In Section 1, we introduce our key concepts of increasing labeled chains in the Bruhat order, 
which generalize Young tableaux, and of skew Schubert polynomials, & w / u (x) for permutations 
u < w. In particular, we show that skew Schubert polynomials are generating functions both 
for Littlewood- Richardson coefficients c™ v for Schubert polynomials (the analog of (1)), and 
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for increasing chains in the Bruhat order (the analog of (2)). Our skew Schubert polynomials 
are different from those defined in [10] (using skew divided difference operators), which do 
not expand with nonnegative coefficients in the basis of Schubert polynomials, in general. 

The skew Schubert polynomial & Wo / w (x) is the ordinary Schubert polynomial & w (x), and 
our formula for & WQ / W (x) in terms of increasing chains is the formula of Corollary 5.3 in [4], 
which inspired us (see the next section for the notation). In Section 2, we give a purely 
combinatorial proof of that formula, relating it to a standard construction [7, 5, 8] in terms of 
rc-graphs by giving a content-preserving bijection between rc-graphs and increasing chains. 

I. Skew Schubert Polynomials 

Let S n be the symmetric group of permutations of {1, 2, . . . , n}. For w G S n , the length, 
£(w), of w is the number of inversions of w. Set w G S n to be the longest permutation, 
w := n . . . 2 1. We use basics on Schubert and Schur polynomials, which may be found in 
any of [14, 15, 9, 16]. 

The main outstanding problem in the theory of Schubert polynomials is the Littlewood- 
Richardson problem [21, Problem 11]: Determine the structure constants c™ v defined by the 
polynomial identity 

e u (x)-e v (x) = J2 c Zv&M- 

w 

Since every Schur polynomial is a Schubert polynomial, this problems asks for the analog 
of the classical Littlewood-Richardson rule. The classical Littlewood-Richardson coefficients 
have important and intricate combinatorial properties [20, Chapter 7, Appendix 1]. The 
Littlewood-Richardson coefficients c™ v for Schubert polynomials should be similarly impor- 
tant. Indeed, they are intersection numbers of Schubert varieties; more precisely, c™ v enumer- 
ates flags in a suitable triple intersection of Schubert varieties indexed by u, v, and w w. 

The cohomology classes a w (called Schubert classes) for w G S n of Schubert varieties form 
an integral basis for the cohomology ring H*¥£ n of the flag manifold. Many identities and 
some formulas for the constants c„ have been obtained by studying the class a u ■ a WQW in 
H*¥£ n [18, 2, 3]. This is because, for u,v,w G £„, we have the following identity in the 
cohomology ring 

w 

@u ' @wqw ' @v C u v (J wo 

and hence, by the duality of the intersection pairing 

(3) o~ u ■ (J WqW = ^ ] c u v (T WQV . 

With this motivation, we would like to define the skew Schubert polynomial & w / u (x) for 
permutations u < w in S n to be the polynomial representative of the class a u • a WQW . Unfor- 
tunately, the cohomology of the flag manifold is isomorphic to the quotient ring 

H*¥£ n := Z[x 1 ,x 2 , ... ,x n ]/ (e^xi, . . . , x n ) \ i = 1, . . . , n) , 

and so there is no well-defined polynomial representative of a u • <J mw . Here ei(xi, . . . ,x n ) is 
the ith elementary symmetric polynomial in x±, . . . , x n . 
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If however we choose the degree reverse lexicographic term order on monomials in the 
polynomial ring Z[xi, x 2 , ■ ■ ■ ,x n ], with X\ < x 2 < ■ ■ ■ < x n , then every element of this 
quotient ring has a unique normal form polynomial representative [22, Prop. 1.1] with respect 
to this term order. These normal form representatives are elements of 

Z ■ {x a | a = (ai, . . . , a n ) with at < n — i} . 

Equivalently, if we set 5 := (n — 1, . . . , 2, 1, 0), then normal form representatives of cohomology 
classes are sums of monomials dividing x 5 . 

Fomin, Gelfand, and Postnikov [6] observed that for w G £ n , the Schubert polynomial 
& w (x) is the normal form representative of the corresponding Schubert class a w . In fact, this 
feature of Schubert polynomials, that one does not need to work modulo an ideal, is what led 
Lascoux and Schiitzenberger to their definition of Schubert polynomial. Thus we define the 
skew Schubert polynomial & w / u (x) to be the normal form representative of the class a u • a WoW . 
These skew Schubert polynomials are generating functions for the coefficients c™ v . 

Theorem 1. For u < w in E n; we have 

(4) & w/u (x) = J2 c Zv<5w v(x). 

V 

Proof. Each side of (4) is the normal form representative of the corresponding side of (3). □ 

While & w / u {x) depends upon n, the Laurent polynomial x~ 5 & w / u {x) does not, as & u (x) and 
x~ 5 & WQW (x) are each independent of n. The independence of & u (x) is the familiar stability 
property of Schubert polynomials. The independence of x~ s & wow (x) may be similarly deduced 
using divided differences. This also follows from Proposition 3, which we prove in Section 2. 

The skew Schubert polynomial & w / u (x) is also a generating function for certain chains from 
u to w in the Bruhat order. The Bruhat order is defined by its covers; u < w if and only 
if u^w is a transposition (k,l) with u(k) < u(l), and for every k < i < I we have either 
u(i) < u(k) or u{l) < u(i). Thus u < u • (k,l) with k < I if u(k) < u(l) and at no position 
between k and / does u take a value between u(k) and u{l). This implies that the difference 
in lengths £(w) — £(u) equals 1. 

The labeled Bruhat order is the labeled reseau (labeled directed multigraph) obtained from 

(k b) 

the Bruhat order by drawing a directed edge u — : — > w whenever u <w with u~ l w = 
where % < k < j and b = u(i) = w(j). We note that all the constructions and statements still 
hold if we uniformly set b = u(j) = w(i), but it is more convenient to use the first definition. 
There will be j—i such directed edges for every cover. This structure was defined in [1] and 
has been crucial in subsequent work on the problem of multiplying Schubert polynomials by 
Bergeron and Sottile. In [2] this structure was called the colored Bruhat order. 

To any (saturated) chain 7 in this reseau, we associate a monomial x 1 in the variables 
xi, . . . ,x n -i, where the power of Xi counts how often i was the first coordinate of a label in 
7. A chain 

/_s (fci,6i) (^2,62) (k m ,b m ) 

(5) u > ui > ■■■ > u m 

in this reseau is increasing if its sequence of labels is increasing in the lexicographic order on 
pairs of integers. 



4 



CRISTIAN LENART AND FRANK SOTTILE 



These definitions allow a nice reformulation of a combinatorial construction of Schubert 
polynomials given in [4]. 

Theorem 2. Let u <w be permutations in S n . Then 

(6) &w/u(x) = ^V/:r 7 , 

the sum over all increasing chains 7 in the labeled Bruhat order from u to w. 

Note that & w (x) = & Wo / w (x). We recover the formula of Bergeron and Sottile for & w (x) [4]. 
Denote by T(w,wo) the set of all increasing labeled chains from w to u>o- 

Proposition 3 (Bergeron-Sottile [4]). Let w G S„. Toen 

6„(x) = £ 

7gr(w,ii;o) 

Theorem 2 has an immediate enumerative consequence. The fa/pe a of a chain 7 in the 
labeled Bruhat order is the (weak) composition a whose ith component counts the number 
of occurrences of % as the first coordinate of an edge label. Thus x 1 = x a . For u < w in the 
Bruhat order and a composition a of £(w) — £(u), let I a (u, w) count the number of increasing 
chains from u to w of type a. Since & WQV (x) = & wo /w v( x ), equating coefficients in (4) and 
using Theorem 2 we obtain 

Corollary 4. I a (u,w) = ^ c™ v I a (w v , w ). 

V 

For a composition a with n— 1 parts where a < 5 coordinatewise, set 

h a (x) := h ai (x 1 )h aa (x 1 ,x 2 )--h an _ 1 (x 1 ,... ,x n -x), 

where h a (xi, . . . ,Xk) is the complete homogeneous symmetric polynomial of degree a in the 
variables x 1: . . . ,x k . We let h a and h a denote the corresponding cohomology classes in H*¥£ n . 
For / G H*¥£ n , set tp a (f) to be the coefficient of a WQ in the product / • h a . This gives a linear 
map ip a : H*¥£ n -> Z. 

Lemma 5. For on?/ / G H*¥£ n , tp a (f) is the coefficient of the monomial x s / x a in the normal 
form representative of f . 

Proof. This follows from two observations. First, both ip a and the map associating to / the 
coefficient of the monomial x 5 /x a in the normal form representative of / are Z-linear maps on 
H*¥£ n . Second, the lemma holds on the basis of Schubert classes, whose representatives are 
Schubert polynomials. This was shown by Kirillov and Maeno [11] and is also a consequence of 
the Pieri formula [18] and the construction of Schubert polynomials given in Proposition 3. □ 

Proof of Theorem 2. Given a chain 7 in the Bruhat order, let end(7) denote its end point. 
The Pieri-type formula for Schubert classes [18] is 

(7) CT u -h a = 0~end( 7 ) (x) , 

the sum over all increasing chains that begin at u, have length a, and whose labels have first 
coordinate k. 
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Applying Lemma 5 to the class a u -a wow shows that the coefficient of the monomial x 5 /x 1 in 
the normal form representative of a w / u is the coefficient of a WQ in the triple product cr u -a WoW -h a , 
where a is the type of the chain 7. Since the coefficient of a wo in a product a v ■ a WQW is the 
Kronecker delta 5^ (by the duality of the intersection pairing), this is the coefficient of a w 
in the product o u • h a . The theorem now follows by expanding this product and iteratively 
applying the Pieri formula. □ 

Remark 6. By the definition of skew Schubert polynomials, we have 

since wqWq = 1 and &i(x) = 1. Thus there should exist a natural bijection between T(l,w) 
and r(woW,Wo) that preserves type. Corollary 4 suggests that, for u < w, this bijection 
should generalize to a map from T(u, w) to the union \J v F(w v, w ) such that the cardinality 
of the inverse image of each chain in T(wqV,Wq) is c™ v . This would give a combinatorial 
interpretation for , and thus solve the Littlewood- Richardson problem. 

Remark 7. The skew Schubert polynomial & w / u (x) also has a geometric interpretation as 
the representative in cohomology of a skew Schubert variety 

X F n X F' 

where X W F. is the Schubert variety whose representative in cohomology is & w (x) and F. and 
F' are flags in general position. Stanley noted that skew Schur polynomials have a similar 
geometric-cohomological interpretation [19, §3]. 

Remark 8. Schubert polynomials & w (x) when w has a single descent at position k (w is a 
Grassmannian permutation) are Schur symmetric polynomials in xi, . . . ,Xk- Even if w and 
u are Grassmannian permutations with the same descent, then it is not necessarily the case 
that the skew Schubert polynomial & w / u (x) equals the corresponding skew Schur polynomial. 
We illustrate this in S 4 , using partitions and skew partitions for indices of Schur polynomials. 
We have the Schubert /Schur polynomials 

( 5i324(^) = x 1 +x 2 = S a (x 1 ,x 2 ), and 

©2413 (X) = x\x 2 +X x x\ = Sp(x 1 ,X 2 ) , 

and also the skew Schur polynomial 

S t p(xi,X 2 ) = XiX 2 + XiX 2 +x\ + x\ 

= Sq(x!,X 2 ) + S m (x!,X 2 ) = 62314(X) + 6l423(a;) • 

On the other hand, ©24i3/i324(^) = ( 5i324(^) • S W0 24i3(a;) and w 2A13 = 3142. By Monk's 
formula (or the Pieri formula (7)), we compute ©2413/1324(2:) to be 

61324(2:) • 63142(2:) = 63241(2:) + 64132(2:) + 63412(2:) 

= 6^2314(2:) + 6^1423(2:) + 6^2143(2:) • 

The first two terms of 62413/1324(2:) carry the same information as the two terms of S [ p(xi,X2), 
but there is also a third term. This is because the multiplication of Schubert polynomials 
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is richer than the multiplication of Schur polynomials. By Theorem 1, this expansion of 
©2413/1324 (2) records the fact that 62413 ( x ) is a summand (with coefficient 1) in each of 

61324(2) ■ ©2314(2;) , ©1324(2) • ©1423(2) , and ©1324(2) ■ 62143(2) • 

Note that the last product (unlike the first two) is not symmetric in x± and X2 even though 
both 61324(2) and 62413(2) are symmetric in x\ and x%. 



RC-graphs are combinatorial objects associated to permutations w in the symmetric group 
S n . They were defined in [7, 1] as certain subsets of 



An rc-graph R is associated to a permutation w G E n if d(R) is a reduced decomposition of 
w. We denote the collection of rc-graphs associated to a permutation w by lZ(w), and the 
permutation corresponding to a given rc-graph R by w(R). 

We represent an rc-graph R as a pseudoline diagram recording the history of the inversions 
of w . To this end, draw n pseudolines going up and to the right such that the zth pseudoline 
begins at position (i, 1) and ends at position (l,w(i)). (The positions are numbered as in a 
matrix, as illustrated in Example 9 below.) The rule for constructing the pseudoline diagram 
is the following: two pseudolines entering at position (i, j) cross at that position if is in 
R, and otherwise avoid each other at that position. Note that two pseudolines cross at most 
once, as d(R) is reduced. 

Conversely, a pseudoline diagram represents a permutation w, where w(i) is the endpoint 
of the pseudoline beginning at position (i, 1). Given a pseudoline diagram where no two 
pseudolines cross more than once, the crossings give an rc-graph associated to the permutation 
represented by the pseudoline diagram. 

Example 9. Here are the pseudoline diagrams of two rc-graphs associated to the permutation 



2. RC-Graphs and Increasing Labeled Chains 



{(k,b) <EZ >0 xZ >0 \k + b<n}. 



We may linearly order such a subset of pairs by setting 




w = 215463. 



1 2 3 4 5 6 



1 2 3 4 5 6 
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Given an rc-graph R, define the monomial 

As shown in [5, 7], the Schubert polynomial & w indexed by w can be expressed as 
(8) G w = xR - 

Comparing this expression with Proposition 3 suggests that there should be a natural bijection 
between 1Z(w) and T(w,w ), where x R ■ x 1 = x s , when R corresponds to 7. Indeed, we give 
such a bijection. 

Given an rc-graph R in lZ(w), we greedily construct the sequence of rc-graphs R = 
R ,Ri,... ,Ri(w )-£(w) as follows. Given Ri, add the pair (k,b) to Ri where (k, b) is the 
pair not in Ri that is minimal in the lexicographic order on pairs. We have the following 
lemma. 

Lemma 10. Given R 6 7l(w), construct the sequence R = R ,Ri, . . . ,Ref Wo )-e(w) as above. 
Then 

1. Every subset Ri is an rc-graph. 

2. w = w(Rq) < w(Ri) < . . . < w(Re( wo yi( w )) = wq is a saturated chain in Bruhat order. 

3. If Ri + i is obtained from Ri by adding the pair (k, b), then in the labeled Bruhat order we 
have the labeled cover 

w(Ri) fc ' 6) > w(R i+ i) . 

4. The labeled chain with the labels of (3) is increasing. 

Example 11. Here is an rc-graph, its pseudoline diagram, and the associated increasing 
labeled chain. 



J 



+ 



1432 



(Li) 



> 4132 



(2,1) 



4231 



(2,2) 



> 4321 



Proof of Lemma 10. We prove the first statement by induction. Suppose that Ri is an rc- 
graph and we add the pair (k, b) to obtain R4+1, where (k, b) is the minimal pair not in R^. 
Consider the pseudoline diagram of Ri near the position (k, b): 



b 



k 




By construction, Ri contains every crossing in the shaded region, so the two pseudolines at 
this position never cross, with one connecting k to b as drawn. Adding the crossing gives a 
new pseudoline diagram for a permutation with exactly one more inversion. Indeed, assume 
that we create a double crossing between one of the two pseudolines obtained from the drawn 
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ones by adding the extra crossing and another pseudoline. It is easy to check graphically that 
this can only happen if the latter pseudoline meets the pseudoline connecting k to b in Ri 
twice. But this is impossible, as Ri is an rc-graph. 

These arguments show that w(Rj) <w(Ri + i), implying the second statement. For the third, 
note that (k, b) is a possible label, and the fourth follows by our greedy algorithm for adding 
pairs not in R. □ 

Observe that the increasing chain 'y(R) constructed in Lemma 10 from an rc-graph R has 
the property that 

(9) if u — > v is a labeled cover in j(R), then u~ l v = (k, I), for some / > k. 

A priori, given that u ^ k,b \ t> is a cover, we are only guaranteed that u~ 1 v = (j, I) with 
j < k < I. We prove that this property holds for any increasing chain ending in w , and also 
that the map 7 l(R) is a bijection from lZ(w) to T(w,w ). This will immediately imply 
Proposition 3. 

Theorem 12. Let wGS„ and 7 be an increasing chain from w to wq. Then 

1. 7 has property (9). 

2. The set of pairs (k, b) with k + b < n and (k, b) not a label of a cover in 7 is an rc-graph. 

Proof. We prove these statements by downward induction on £(w). They hold trivially when 
w = w . Consider the first two steps in 7: 

(10) w ► v > ■ ■ ■ 

Since 7 is increasing, we have j < k or j = k and a < b. Since the part of the chain beginning 
with v is an increasing chain, our inductive hypothesis implies that subsequent covers involve 
only positions of v greater than or equal to k. Thus if i < k, we must have v(i) = n + 1 — i 
and v(k) = b. 

Suppose that w~ l v = (i, I) so that w(i) = a = v(l) and % < j < I. We consider cases i = k, 
i = k — 1, and % < k — 1 for % separately, showing that in each case, we have j = i. 
If % = k, then, as % < j < k, we must have j = k = i. 

If % < k — 1, then the betweenness condition for covers in the Bruhat order and the values 
of v on 1, 2, ... , k— 1 force I — i + 1 and so again we have j = i. 

Now suppose that i — k — 1 and j 7^ i so that j — k and we then have a < b. Then 

v(i) = v(k-l) = n+2-k > v(k) = b > a = v (I) , 

and so the betweenness condition on covers w < v is violated. This completes the proof of 
Statement I. 

For the second statement, consider the procedure of successively removing labels (k, b) of 
covers of 7 from the rc-graph of Wq (which contains all crossings). Suppose that this process 
applied to the chain (10) above v has created an rc-graph R associated to v. We argue that 
removing the crossing (j, a) from R creates an rc-graph associated to w. First, (j, a) is a 
crossing of R, as we have only removed crossings that exceed (j, a) in the lexicographic order. 
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Consider the pseudoline diagram of R near (j, a), which we represent on the left. 

a a 




Note that R contains all crossings in the shaded region. Removing the crossing at (j, a) creates 
the picture on the right. Arguing as in the proof of Lemma 10 (but in reverse) shows that we 
obtain an rc-graph associated to w. □ 

Remark 13. Property (9) of increasing chains from w to Wq implies that the branching in the 
tree of increasing chains from w to w is quite simple; it only depends upon the permutation 
at a node and not on the history of the chain. The branches at a permutation u consists of 

(k,b) 

all possible covers u — — > u(k, I), where k is the minimal position i such that u(i) + % < n + 1 
and b = u(i) and I is any position greater than k such that u(l) > b = u(k) and if k < i < I, 
then u(i) is not between u(k) and u{l). 

We may order the branches at u by the position /. This leads to an efficient lexicographic 
search of this tree for generating all such increasing chains, and thus all rc-graphs in lZ(w) 
as well as the multiset of monomials in the Schubert polynomial & w (x). The computational 
complexity and memory usage of this algorithm depend only upon n (the number of entries 
in w), the length l(w) of w, and the number of increasing chains, c := & w (l, 1, . . . , 1). This 
is true, in fact, for any algorithm generating c combinatorial objects corresponding to the 
monomials in the Schubert polynomial & w (x). Furthermore, it is not hard to see that the 
above algorithm is of order 0(nlc), where I := l(w ) — l(w) = (™) — l(w). Indeed, note that 
the number of internal nodes of a rooted tree with a fixed number of leaves (c in our case) and 
a fixed distance from its root to all its leaves (I in our case) is maximized by the tree with all 
internal nodes having only one descendant; thus, there are 0(lc) nodes. On the other hand, 
finding the descendants of a given node is of order 0{n). 

Let us also note that a closely related algorithm, with the same complexity as the one 
above, appears in [12, 17]; it is described in terms of removing crossings from the rc-graph 
corresponding to wq. More precisely, the authors of the mentioned papers use combinatorial 
versions of the divided difference operators in order to recursively generate the rc-graphs 
corresponding to a permutation w. One starts from w and applies these operators using any 
chain in the weak order on S n from u> to w. A certain choice of chain guarantees that each 
rc-graph in the obtained tree has at least one descendant upon applying the corresponding 
operator. It turns out that the rc-graphs on a given level i in this tree correspond (via the 
bijection described above) to the subchains starting at level / — i in the tree constructed by 
the previous algorithm. 

While this algorithm for generating the multiset of monomials in the Schubert polynomial 
& w (x) also computes this polynomial, A. Buch has pointed out to us that the transition 
formula of Lascoux and Schiitzenberger [14] provides a more efficient algorithm to compute 
& w (x). 

We conclude by mentioning the connection with the insertion algorithm for rc-graphs in 
[13], which bijectively proves the Pieri formula (7) for multiplying a Schubert polynomial by a 
complete homogeneous symmetric polynomial hi(xi, ... , x^). Given an rc-graph R with x R = 
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x" 1 . . . x"-i, the chain j(R) can be obtained by successive insertions into R corresponding to 
multiplications of x R by the monomials x^~ k ~ ak in fo n _fc— ak [x\ ) . . . , for k — 1, . . . , n 1. 
In this this special case, the insertion algorithm reduces to the older one in [1]. 

We thank Ezra Miller for his detailed comments on an earlier version of this paper. 
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